Path Integrals for Continuous-Time Markov Chains
نویسنده
چکیده
This note presents a method of evaluating the distribution of a path integral for Markov chains on a countable state space. We illustrate the method with reference to several models, including birth-death processes and the birth, death and catastrophe process.
منابع مشابه
Integrals for continuous-time Markov chains.
This paper presents a method of evaluating the expected value of a path integral for a general Markov chain on a countable state space. We illustrate the method with reference to several models, including birth-death processes and the birth, death and catastrophe process.
متن کاملAn Extension of Peskun and Tierney Orderings to Continuous Time Markov Chains
Peskun ordering is a partial ordering defined on the space of transition matrices of discrete time Markov chains. If the Markov chains are reversible with respect to a common stationary distribution π, Peskun ordering implies an ordering on the asymptotic variances of the resulting Markov chain Monte Carlo estimators of integrals with respect to π. Peskun ordering is also relevant in the framew...
متن کاملStochastic Dynamic Programming with Markov Chains for Optimal Sustainable Control of the Forest Sector with Continuous Cover Forestry
We present a stochastic dynamic programming approach with Markov chains for optimal control of the forest sector. The forest is managed via continuous cover forestry and the complete system is sustainable. Forest industry production, logistic solutions and harvest levels are optimized based on the sequentially revealed states of the markets. Adaptive full system optimization is necessary for co...
متن کاملApproximate Symbolic Model Checking of Continuous-Time Markov Chains
This paper presents a symbolic model checking algorithm for continuous-time Markov chains for an extension of the continuous stochastic logic CSL of Aziz et al 1]. The considered logic contains a time-bounded until-operator and a novel operator to express steady-state probabilities. We show that the model checking problem for this logic reduces to a system of linear equations (for unbounded unt...
متن کاملEstimation of the Entropy Rate of ErgodicMarkov Chains
In this paper an approximation for entropy rate of an ergodic Markov chain via sample path simulation is calculated. Although there is an explicit form of the entropy rate here, the exact computational method is laborious to apply. It is demonstrated that the estimated entropy rate of Markov chain via sample path not only converges to the correct entropy rate but also does it exponential...
متن کامل